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-Abstract: Research has shown the potential of a problem-based approach to enhance 

students’ learning. The interactive nature of hypermedia technology and its ability to 
deliver information in different media formats can provide unique capabilities for 
implementing problem-based learning (PBL) environments. Yet, we know little about the 
types of tools that are effective in supporting learning in hypermedia supported PBL 
environments. The purpose of this exploratory study is to investigate both the use of tools 
and design features in a piece of PBL software and their effectiveness on middle school 
students’ learning. Since data gathering has just been completed, there has not been 
sufficient time to complete the data analysis. Data analysis will be completed and research 
results and conclusions presented in the conference presentation. 
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1. Research Framework 

1.1 Problem-based Learning 

There has been a growing body of research on authentic and situated learning environments utilizing 
the problem-based approach to learning [Cognition and Technology Group at Vanderbilt, 1992; Pedersen, 
Williams, 1999]. Problem-based learning (PBL) emphasizes solving authentic problems in authentic 
contexts. It is an approach where students are given a problem, replete with all the complexities typically 
found in real world situations, and work collaboratively to develop a solution. Problem-based learning 
provides students an opportunity to develop skills in problem definition and problem solving, to reflect on 
their own learning, and develop a deep understanding of the content domain learning [Cognition and 
Technology Group at Vanderbilt 1994; Jacobson & Spiro, 1995; Lajoie 1993]. This approach was developed 
in the sixties for medical education, and has since been used in various subject areas, such as business, law, 
education, architecture, and engineering. 

Literature has shown that problem-based learning can facilitate the improvement of student attitude 
toward the content area learning [Cognition and Technology Group at Vanderbilt, 1992; Williams, 1999]. 
Explanations offered for this are that students perceive the relevance of the work [Barrows 1986] or compare 
the task of finding information and developing a solution to solving a mystery [Cognition and Technology 
Group at Vanderbilt 1992]. Medical students using a PBL curriculum have been shown to be able to pursue 
learning independently better than their peers receiving a traditional curriculum [Aspy, Aspy, & Quinby 
1993], a finding which supports the claim that PBL prepares students to become independent, lifelong 
learners. High school students in a semester long class which used PBL exclusively showed a significant 
increase in their spontaneous use of one problem-solving step: problem finding, the ability to identify and 
formulate problem statements [Gallagher, Stepien, & Rosenthal 1992]. Similarly, students in a PBL 
curriculum have shown a greater ability than students in traditional classes to break complex problems into 
their component parts and identify sub-problems which must be solved in order to solve the main problem 
[Cognition and Technology Group at Vanderbilt 1992; Gallagher & Stepien 1996]. 

In recent years, there has been a growing interest among educators to use PBL in the K-12 setting. 
However, the experiences of college level programs and early efforts in K-12 schools have shown PBL to be a 
particularly demanding instructional approach for both students and teachers [Barrows & Tamblyn, 1980; 
Cognition and Technology Group at Vanderbilt, 1992]. Students must perform a wide variety of tasks with 
which they may have had limited prior experience. These include problem finding, hypothesis generation, 
identification of learning needs, location of resources to meet these learning needs, data collection and 
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organization, and development of a solution plan supported by evidence and reasoning. Teachers, in their 
role as facilitators, are responsible to provide support for this wide variety of activities to students who may 
vary greatly in their needs. Because the variability of the classroom teacher’s ability to function in this role 
has a profound effect on the success of PBL [Walton & Matthews 1989], instructional materials that provide 
some of this support seem warranted. 

1.2 Hypermedia and Problem-based Learning 

The interactive nature of computers and the ability to deliver information in different media formats 
provides unique capabilities for the implementation of PBL environments. In particular, the capabilities of 
hypermedia have much to offer designers of advanced learning environments. Hypermedia is a rich 
environment containing information in many different forms, including text, graphics, audio, video, and 
animation. The student is not forced to access the resources in any predetermined order, but can navigate 
within the environment in a nonlinear fashion [Burton, Moore, & Holmes 1995]. The support of multiple 
media types and the possibility of nonlinear navigation are particularly useful in the creation of computer- 
based PBL environments. 

The nonlinear nature of hypermedia is consistent with the characteristics of PBL. As students are 
engaged in PBL, they need to collect data and access resources. This suggests a high degree of control on the 
part of the learner. Hypermedia can support this by allowing students to access needed resources at the time 
it is most appropriate. Hypermedia allows students to have more control over their learning. They become 
actively engaged in decision-making while traversing the environment. Research on learner control versus 
program control in hypermedia environments suggests that subjects under learner control score higher than 
those under program control on achievement posttest and have a more positive attitude toward learning 
[Hannafin & Sullivan 1995]. 

Though PBL can be implemented with traditional media, hypermedia provides unique capabilities 
for its implementation. The nonlinear nature of hypermedia allows students to explore the PBL environment 
accessing resources as the need arises. Hypermedia can also facilitate the development of authenticity in the 
learning environment. Williams [Williams 1992] suggests that law and medical school curricula could be 
improved by the use of hypermedia environments to engage students in authentic activities within an 
authentic context. 

1.3 Hypermedia and Cognitive Tools 

When working in everyday situations, individuals use tools and resources such as computers, 
calculators, concepts, and formulas in order to solve problems. Therefore, tools must be considered when 
creating authentic environments for student learning. A tool does not necessarily have to be a tangible object. 
For example, an engineer may use a mathematical formula to calculate the area of a cylinder. Different 
disciplines and professions may use ideas and tools in very different ways. 

Many researchers argue that cognitive tools can support and enhance student learning [Lajoie, 1993]. 
These tools come in many forms and can support students in a variety of tasks they must perform as they 
engage in problem solving. Hypermedia has the potential to make these tools readily available to students. 

As discussed earlier, PBL occurs within a context where knowledge is naturally situated. Tools can 
be employed to create an authentic context in which students can work. The Adventures of Jasper Woodsbury 
series utilizes video-based scenarios in order to create a context for learning [Cognition and Technology 
Group at Vanderbilt, 1992]. The video segment provides a focus for the learning activity and may also help 
students who are poor readers. Hypermedia can also present a scenario, but has the advantage of allowing 
students to explore the environment in which the scenario is set. 

Hypermedia can augment memory and support students in reflecting on their problem-solving 
process. Sherlock /, software that creates an environment in which Air Force technicians practice avionics 
troubleshooting, includes tools to support cognitive processing [Lajoie, 1993]. Avionics troubleshooting is a 
complex task requiring the technician to entertain many variables and remember a series of completed tests. 
Sherlock I allows the student to view the steps he or she has taken in troubleshooting a problem. The ability 
to view the solution path supports students in reflection on the problem solving process without the need for 
them to rely on their recollection of every step. The software-generated problem solving steps also make 
explicit student thinking, an essential component in stimulating metacognitive awareness. Likewise, 
reflective questions can also be effective in promoting metacognitive thinking. 
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Hypermedia environments can offer a comprehensive set of resources to enable students to meet 
their learning needs. In order for students to engage in problem solving, they must have access to 
information. Information can be provided to students in a number of tools. The Lab Design Project (LDP), 
which allows students to actively take part in sociological research concerning a fictitious biotechnology 
building, permits students to gather information from such diverse formats as interviews with employees, 
building plans, letters, and sketches [Honebein 1996]. 

Hypermedia can provide electronic notebooks, which, in addition to providing space for student note 
taking, can include advanced features to help support the student in constructing meaning. The notebook in 
Bio-world contains a section that displays the students’ previous actions such as database searches and 
diagnoses for patients [Lajoie 1993]. These features help support the student’s memory and metacognitive 
thinking. Other projects have augmented the traditional notebook with the ability to support multimedia, 
collaboration among students, and the ability to create links between concepts [Edelson, Pea, Gomez 1996]. 

Though literature supports the efficacy of problem-based learning, little research exists which 
investigates the types of tools or features that are effective in supporting students working in PBL 
environments. PBL software is beginning to find its way into schools. However, the relative effectiveness of 
the various tools incorporated in these programs has yet to be studied. In order to design an effective 
computer-supported PBL learning environment, it is important to understand the tools and design features 
included in the software, and their impact on learning. It is, therefore, the purpose of this study to examine 
and understand how middle school students use and interact with a piece of recently available computer- 
supported PBL software. 

2 Research Questions 

This study investigates the use of tools and design features as employed in a piece of problem-based 
learning software and their effectiveness on middle school students’ learning of science concepts. 
Specifically, the following research questions formed the focus of the study: 

(1) Do any navigational profiles emerge among the student teams, and if so, how do they compare in 

usage of the tools embedded in the PBL environment? 

(2) If navigational profiles exist how do they compare in terms of the external criteria: science aptitude, 

recall and transfer measures, and attitude toward the environment? 

(3) If navigational profiles do not exist, what is the relationship between tool use and student 

achievement and attitude? 

3. Design of Study 

3.1 Sample 

The participants of the study (N = 100) were students enrolled in sixth grade science at a middle 
school located in a medium-sized city in the southwestern United States. The age*of the students ranged from 
12-14 years. There were approximately an equal number of male and female students. Students worked in 
teams of heterogeneous groups determined by the classroom teacher. 

3.2 Treatment 

For this research, a hypermedia-supported problem-based learning environment was developed in the 
content area of astronomy. The learning goals of the HALE were for students to be able to: 

• Plan and implement procedures for solving complex problems 

• Identify relevant information needed in solving a complex problem 

• Identify the characteristics of objects in our solar system 

• Describe components that comprise probes used for astronomical research 

• Rationalize the design of a probe in regards to it’s intended mission 

• Analyze data and draw conclusions from astronomical data 

The environment supports students with the following tools: notebook, bookmarks, presentation 
tool, probe builder, solar system database, mission database, alien database, and expert tool. 
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3.3 Dependent Measures 

Of primary interest in this study is how students of varying characteristics working in teams utilize 
the tools and support present in the PBL environment and how those tools affect student learning. Therefore, 
all student actions in the environment were logged to a data file for analysis. Additionally, measures of 
knowledge recall and transfer were administered, and interviews were conducted with the participants and the 
classroom teacher. The triangulation of the quantitative and qualitative data sources will help to answer the 
research questions with richer and more detailed information. 

3.3.1 Audit Trail of Tool Access 

The learning environment kept a log of all student actions in the environment, notebook contents, 
probes built, and time spent using the program. The log consists of time and date stamped entries. 

3.3.2 Recall Measure 

The recall measure evaluated the amount of declarative knowledge students have concerning the 
astronomy concepts being taught in the study. The measure consists of multiple choice and fill-in-the-blank 
items. A panel of reviewers were asked to verify the validity of the measure. Items were revised until 
agreement was reached. The recall instrument was administered as a pretest, posttest, and retention measure. 
Due to the factual and objective nature of the measure, it was scored by the principal investigators. The recall 
measure was used to gauge the degree to which students acquire an understanding of astronomy. 

3.3.3 Attitude Toward Science in School Assessment 

Attitude toward science was measured with Germann’s (1988) Attitude Toward Science in School 
Assessment (ATSSA). Germann developed ATSSA in response to the ambiguity that exists in science 
attitude as a construct. Germann points out that there are many dimensions that make up attitude toward 
science and that attitude measures must be clear as to what they measure. Attitude toward science can 
include such things as scientific careers, methods of teaching science, scientific interests, assessing ideas and 
information, and science as a subject. The purpose of ATSSA is to measure a single dimension of attitude 
toward science, that is, how students feel about science as a subject (Germann). The instrument consists of 15 
Likert scale items and has a Cronbach alpha of 0.95. 

3.3.4 Attitude Toward the Environment 

The Attitude Response Log gathered information on students’ attitudes toward the difficulty of the 
learning environment, level of interest in the activity, attitude toward learning science with computer, and 
attitude toward the tools. The questionnaire used in this study consisted of 20 likert-scale items. The 20 
items are partitioned into statements concerning the following areas: how students perceived the difficulty of 
the activity (difficulty scale), what students’ level of interest was in the activity (interest scale), whether 
students prefer learning science on computer or in class with other students (computer preferences scale), and 
how students liked the tools (tool scale). 

3.3.5 Transfer Measures 

As noted in the literature review section, students are often unable to apply what they have learned in 
problem solving situations. The review of literature put forth that this is due to the decontextualization of 
learning. Therefore, in order to evaluate student learning it is not sufficient to only measure acquisition of 
concepts. But one must also measure the ability to apply that knowledge in context. Hence, near and far 
transfer measures have been included in the design of this research. 

3.3. 5.1 Description of Near Transfer Measure 

In order to evaluate to what degree students are able to apply their knowledge in solving problems 
similar to the one received in the treatment, participants were given a scenario in textual form describing a 
problem similar to the one received during the study. The near transfer measure was given as a posttest and 
used to provide information on the effects of the expert stories on student learning. . After reading the 
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problem scenario, participants were asked to provide the following information: hypothesis, rationale, list of 
information needed to solve the problem, and a description of the tools needed to solve the problem. 

3.3.5.2 Description of Far Transfer Measure 

In order to evaluate to what degree students are able to apply their knowledge in solving problems 
that they have not encountered before, students were asked to solve a problem unlike those encountered 
during the treatment. The far transfer measure was given as a posttest and used to gauge the degree to which 
students are able to apply what they have learned to a novel situation. . After reading the problem scenario, 
participants were asked to provide the following information: hypothesis, rationale, list of information needed 
to solve the problem, and a description of the tools needed to solve the problem. 

3.3.5.3 Scoring of Near and Far Transfer Measures 

Since scoring the near and far transfer measures is subjective in nature, responses from the near and 
far transfer measures were evaluated by three trained graders who will be blind to the subjects’ treatment 
conditions. Due to the large number of responses to evaluate, the responses will be randomly divided 
between the three graders. In order to ensure inter-rater reliability, each grader was given an additional 20% 
of the essays from each of the other two graders. These additional essays were randomly selected, scored, 
and compared with the scores from all graders. Each essay was scored in the following manner: 

1. A score from 0 to 15 was given for the essay on the following criteria: 
(a) Quality/plausibility of the hypothesis and supporting rationale, (b) Level of incorporation of 
details and facts in the essay, (c) Thoroughness of analysis of needed information to support answer, 
(d) Number of tools cited and the appropriate use of them, (e) overall originality of ideas, (f) overall 
quality of the essay. 

2. Scores from each of the criteria were collapsed into one overall score for the response. 

3.4 Procedures 

Data was collected from ten intact classes resulting in data from 60 student groups. Students were 
presented with information about the nature of the learning environment. They were informed that at the end 
of the activity they would be asked for input concerning the aspects of the environment they found useful and 
those they did not. 

Prior to treatment, students were asked to complete the recall measure and attitude toward learning 
science instrument. After all groups had completed the activity, participants were asked to complete the 
recall measure, near transfer measure, and far transfer measure and attitude toward environment instrument. 
Three weeks after the completion of the study, the recall measure was administered once again to all of the 
participants to collect retention data. 

3.5 Analyses 

Since data gathering has just been completed, there has not been sufficient time to complete the data 
analysis. Data analysis will be completed and research results and conclusions presented in the conference 
presentation. In several studies of hypermedia, researchers analyzed navigational data and found that 
different profiles of users tended to emerge. In this study, the users were students working in heterogeneous 
teams of two to three students determined by the classroom teacher. Each team used the HALE as a group, 
and hence we will analyze the group navigational data. Individual navigational performance is difficult to 
record, given that the students used the learning environment as a group. To determine whether different 
group profiles emerge, we will analyze the group navigational data, using cluster analysis. If clusters do 
emerge, we can study how each cluster tends to use the different navigational tools and design features, 
giving us a profile for each cluster. As other authors have done in similar studies, the external variables will 
be compared to the profile groupings using analysis of variance procedures. If differences do exist, this will 
not only lend support to the existence of such profiles but also give insight into whether learning as expressed 
by the recall and transfer measures is different among the profiles. 

If no clusters emerge, we will look for relationships between the group measures of navigational variables and 
the individual student measures of the external criteria. Pearson correlation coefficients and analysis of 
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variance procedures will be used to find these relationships. Here the heterogeneity of the group may hinder 
the analysis. For example, if one group was composed of students of varying levels of ability, the 
navigational scores would be the same for all group members whereas their recall and transfer measures may 
range from high to low. The resulting correlation coefficients may be insignificant. In this scenario, future 
studies of learning environment may need to take into account an individual’s navigation through the 
program. 

4. Conclusions 

As problem-based learning becomes more popular, instructional designers must find new ways to 
utilize technology in order to support problem-based learning. For this research an innovative problem-based 
learning environment was created which provided the context for learning and authentic activities for students 
to engage. Once the analysis is completed, it should provide insight into the tools and features that can 
support students of varying characteristics when working in hypermedia-supported problem-based learning 
environments. 
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